Display system with time interleaving

ABSTRACT

An electronic device such as a head-mounted device may have a display that produces a display image. The head-mounted device may have an optical system that merges real-world images from real-world objects with display images. The optical system provides the real-world images and display images to an eye box for viewing by a user. The optical system may use time interleaving techniques and/or polarization effects to merge real-world and display images. Switchable devices such as polarization switches and tunable lenses may be controlled in synchronization with frames of display images. Geometrical phase lenses may be used that exhibit different lens powers to different polarizations of light.

This application claims the benefit of provisional patent applicationNo. 62/886,171, filed Aug. 13, 2019, which is hereby incorporated byreference herein in its entirety.

BACKGROUND

This relates generally to electronic devices and, more particularly, toelectronic devices with optical systems for merging display content andreal-world content.

Electronic devices sometimes include displays. For example, wearableelectronic devices such as head-mounted devices may include displays fordisplaying computer-generated content that is overlaid on real-worldcontent. An optical system is used to merge real-world content anddisplay content.

Challenges can arise in providing satisfactory optical systems formerging real-world and display content. If care is not taken, issues mayarise with optical quality and other performance characteristics.

SUMMARY

An electronic device such as a head-mounted device may have a displaythat produces a display image. The head-mounted device may have anoptical system through which a user with eyes in eye boxes may viewreal-world objects. During operation, the optical system may be used tomerge real-world images from real-world objects with display images.

A display may produce images in frames. Different objects may bedisplayed in alternating image frames. The optical system may beadjusted in synchronization with the alternating image frames to displaythe different objects at different focal planes.

In some configurations, the optical system may have an intensity switchformed from a pair of linear polarizers and an interposed polarizationswitch. The polarization switch may be operated in a first state inwhich linearly polarized light of a given polarization is not rotated bythe polarization switch and a second state in which the linearlypolarized light of the given polarization is rotated by 90°.

Additional components may be incorporated in the optical system such asfront and rear bias lenses with complementary lens powers, apolarization switch for helping to merge real-world images and displayimages in a time interleaved fashion, and geometrical phase lenses thatpresent different lens powers to images with different polarizations.Tunable lenses may be used to place display images at differentrespective focal plane distances from the eye boxes.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an illustrative electronic device suchas a head-mounted display device in accordance with an embodiment.

FIG. 2 is a top view of an illustrative head-mounted device inaccordance with an embodiment.

FIGS. 3A and 3B are cross-sectional views of illustrative opticalsystems with time interleaving and tunable lenses in accordance withembodiments.

FIGS. 4A, 4B, 5A. 5B, and 6-9 are cross-sectional side views ofillustrative optical systems with geometrical phase lenses in accordancewith an embodiment.

DETAILED DESCRIPTION

Electronic devices may include displays and other components forpresenting content to users. The electronic devices may be wearableelectronic devices. A wearable electronic device such as a head-mounteddevice may have head-mounted support structures that allow thehead-mounted device to be worn on a user's head.

A head-mounted device may contain a display for displaying visualcontent to a user. The head-mounted device may also include an opticalsystem that helps a user view real-world objects while viewing displaycontent. The optical system may include optical components that mergereal-world image light with image light associated with images that aredisplayed by the display. When both real-world image light and displayimage light are visible to a user, the head-mounted device may placecomputer-generated objects within the physical environment surrounding auser.

Real-world content may be merged with display content usingtime-division multiplexing, polarization multiplexing, and/or otherarrangements for combining light from real-world objects with light fromdisplays.

A schematic diagram of an illustrative system that may includehead-mounted device with an optical system for merging real-worldcontent with display content is shown in FIG. 1 . As shown in FIG. 1 ,system 8 may include one or more electronic devices such as electronicdevice 10. The electronic devices of system 8 may include computers,cellular telephones, head-mounted devices, wristwatch devices, and otherelectronic devices. Configurations in which electronic device 10 is ahead-mounted device are sometimes described herein as an example.

As shown in FIG. 1 , electronic devices such as electronic device 10 mayhave control circuitry 12. Control circuitry 12 may include storage andprocessing circuitry for controlling the operation of device 10.Circuitry 12 may include storage such as hard disk drive storage,nonvolatile memory (e.g., electrically-programmable-read-only memoryconfigured to form a solid-state drive), volatile memory (e.g., staticor dynamic random-access-memory) etc. Processing circuitry in controlcircuitry 12 may be based on one or more microprocessors,microcontrollers, digital signal processors, baseband processors, powermanagement units, audio chips, graphics processing units, applicationspecific integrated circuits, and other integrated circuits. Softwarecode may be stored on storage in circuitry 12 and run on processingcircuitry in circuitry 12 to implement control operations for device 10(e.g., data gathering operations, operations involving the adjustment ofthe components of device 10 using control signals, etc.). Controlcircuitry 12 may include wired and wireless communications circuitry.For example, control circuitry 12 may include radio-frequencytransceiver circuitry such as cellular telephone transceiver circuitry,wireless local area network (WiFi®) transceiver circuitry, millimeterwave transceiver circuitry, and/or other wireless communicationscircuitry.

During operation, the communications circuitry of the devices in system8 (e.g., the communications circuitry of control circuitry 12 of device10) may be used to support communication between the electronic devices.For example, one electronic device may transmit video and/or audio datato another electronic device in system 8. Electronic devices in system 8may use wired and/or wireless communications circuitry to communicatethrough one or more communications networks (e.g., the internet, localarea networks, etc.). The communications circuitry may be used to allowdata to be received by device 10 from external equipment (e.g., atethered computer, a portable device such as a handheld device or laptopcomputer, online computing equipment such as a remote server or otherremote computing equipment, or other electrical equipment) and/or toprovide data to external equipment.

Device 10 may include input-output devices 22. Input-output devices 22may be used to allow a user to provide device 10 with user input.Input-output devices 22 may also be used to gather information on theenvironment in which device 10 is operating. Output components indevices 22 may allow device 10 to provide a user with output and may beused to communicate with external electrical equipment.

As shown in FIG. 1 , input-output devices 22 may include one or moredisplays such as display(s) 14. In some configurations, display 14 ofdevice 10 includes left and right display devices (e.g., left and rightcomponents such as left and right scanning mirror display devices,liquid-crystal-on-silicon display devices, digital mirror devices, orother reflective display devices, left and right display panels based onlight-emitting diode pixel arrays (e.g., organic light-emitting displaypanels or display devices based on pixel arrays formed from crystallinesemiconductor light-emitting diode dies), liquid crystal display panels,and/or or other left and right display devices in alignment with theuser's left and right eyes, respectively. In other configurations,display 14 includes a single display panel that extends across both eyesor uses other arrangements in which content is provided with a singlepixel array.

Display 14 is used to display visual content for a user of device 10.The content that is presented on display 14 may include virtual objectsand other content that is provided to display 14 by control circuitry 12and may sometimes be referred to as computer-generated content, displaycontent, display images, display light, etc. Computer-generated contentmay be displayed in the absence of real-world content or may be combinedwith real-world content. In some configurations, a real-world image maybe captured by a camera (e.g., a forward-facing camera) so thatcomputer-generated content may be electronically overlaid on portions ofthe real-world image (e.g., when device 10 is a pair of virtual realitygoggles with an opaque display). In other configurations, an opticalsystem (e.g., an optical coupling system) may be used to allowcomputer-generated content to be optically overlaid on top of areal-world image. As an example, device 10 may have a see-throughdisplay system that provides a computer-generated image to a userthrough a beam splitter, prism, holographic coupler, or other opticalcoupler while allowing the user to view real-world objects through theoptical coupler.

Input-output circuitry 22 may include sensors 16. Sensors 16 mayinclude, for example, three-dimensional sensors (e.g., three-dimensionalimage sensors such as structured light sensors that emit beams of lightand that use two-dimensional digital image sensors to gather image datafor three-dimensional images from light spots that are produced when atarget is illuminated by the beams of light, binocular three-dimensionalimage sensors that gather three-dimensional images using two or morecameras in a binocular imaging arrangement, three-dimensional lidar(light detection and ranging) sensors, three-dimensional radio-frequencysensors, or other sensors that gather three-dimensional image data),cameras (e.g., infrared and/or visible digital image sensors), gazetracking sensors (e.g., a gaze tracking system based on an image sensorand, if desired, a light source that emits one or more beams of lightthat are tracked using the image sensor after reflecting from a user'seyes), touch sensors, buttons, capacitive proximity sensors, light-based(optical) proximity sensors, other proximity sensors, force sensors,sensors such as contact sensors based on switches, gas sensors, pressuresensors, moisture sensors, magnetic sensors, audio sensors(microphones), ambient light sensors, microphones for gathering voicecommands and other audio input, sensors that are configured to gatherinformation on motion, position, and/or orientation (e.g.,accelerometers, gyroscopes, compasses, and/or inertial measurement unitsthat include all of these sensors or a subset of one or two of thesesensors), radio-frequency sensors that determine the location of otherdevices (and therefore the relative position of such devices relative todevice 10), and/or other sensors.

User input and other information may be gathered using sensors and otherinput devices in input-output devices 22. If desired, input-outputdevices 22 may include other devices 24 such as haptic output devices(e.g., vibrating components), light-emitting diodes and other lightsources, speakers such as ear speakers for producing audio output, andother electrical components. Devices 24 may include one or moreadjustable optical components such as liquid crystal devices or otherelectrically adjustable optical components. These components may formpolarization switches. Polarization switches, which may sometimes bereferred to as electrically tunable wave plates or electricallycontrollable polarization rotators, may be adjusted to rotate linearlypolarized light by different amounts (e.g., 0° or 90° depending on thestate of the switch). If desired, a polarization switch may be used witha pair of polarizers to form an electrically adjustable shutter(sometimes referred to as a light modulator or intensity switch). Ifdesired, devices 24 may include tunable lenses. Tunable lenses may beformed from liquid crystal devices and other electrically adjustabledevices. Tunable lenses may be adjusted to produce different lens powers(e.g., desired positive and/or negative lens powers) and/or to adjustthe lateral location of the lens center (e.g., to accommodate differentuser gaze directions). For example, tunable lenses can be adjusted tomove the position of the centers of the lenses based on informationgathered in real time from a gaze detection system.

If desired, device 10 may include circuits for receiving wireless power,circuits for transmitting power wirelessly to other devices, batteriesand other energy storage devices (e.g., capacitors), joysticks, buttons,and/or other components.

Electronic device 10 may have housing structures (e.g., housing walls,straps, etc.), as shown by illustrative support structures 26 of FIG. 1. In configurations in which electronic device 10 is a head-mounteddevice (e.g., a pair of glasses, goggles, a helmet, a hat, etc.),support structures 26 may include head-mounted support structures (e.g.,a helmet housing, head straps, temples in a pair of eyeglasses, gogglehousing structures, and/or other head-mounted structures). Thehead-mounted support structures may be configured to be worn on a headof a user during operation of device 10 and may support display(s) 14,sensors 16, other components 24, other input-output devices 22, andcontrol circuitry 12.

FIG. 2 is a top view of electronic device 10 in an illustrativeconfiguration in which electronic device 10 is a head-mounted device. Asshown in FIG. 2 , electronic device 10 may include support structures 26that are used in housing the components of device 10 and mounting device10 onto a user's head. These support structures may include, forexample, structures that form housing walls and other structures for amain unit (e.g., support structures 26-2) and additional structures suchas straps, temples, or other supplemental support structures (e.g.,support structures 26-1) that help to hold the main unit and thecomponents in the main unit on a user's face so that the user's eyes arelocated within eye boxes 60.

Display 14 may include left and right display portions (e.g., sometimesreferred to as left and right displays, left and right display devices,left and right display components, or left and right pixel arrays). Anoptical system for device 10 may be formed from couplers 84 (sometimesreferred to as input couplers), waveguides 86, and an optical systemformed from one or more optical components such as components 100 and102. Components 100 may be interposed between the front (outwardlyfacing) side of device 10 and waveguides 86 (e.g., between real-worldobject 90 and waveguides 86). Components 102 may be interposed betweenwaveguides 86 and the rear (inwardly facing) side of device 10 (e.g.,between waveguides 86 and eye boxes 60. Components 100 and 102 mayinclude fixed and/or adjustable components that help placecomputer-generated content at a desired focal plane and that help mergethis content with real-world image light that is passing throughcomponents 10 and 102 and waveguide 86 to eye boxes 60. A user with eyeslocated in eye boxes 60 may view real-world objects through the opticalsystem formed from components 100, waveguide 86, and components 102 andother components of device 10 while viewing overlaid computer-generatedcontent from display 14.

As shown in FIG. 2 , the left portion of display 14 may be used tocreate an image for a left-hand eye box 60 (e.g., a location where aleft-hand image is viewed by a user's left eye). The right portion ofdisplay 14 may be used to create an image for a right-hand eye box 60(e.g., a location where a right-hand image is viewed by a user's righteye). In the configuration of FIG. 2 , the left and right portions ofdisplay 14 may be formed by respective left and right display devices(e.g., digital mirror devices, liquid-crystal-on-silicon devices,scanning microelectromechanical systems mirror devices, other reflectivedisplay devices, or other displays).

Optical couplers 84 (e.g., prisms, holograms, etc.) may be used tocouple respective left and right images from the left and right displayportions into respective left and right waveguides 86. The images may beguided within waveguides 86 in accordance with the principal of totalinternal reflection. In this way, the left and right images may betransported from the left and right sides of device 10 towards locationsin the center of device 10 that are aligned with left and right eyeboxes 60. Waveguides 86 may be provided with respective left and rightoutput couplers 88 such as holograms formed on or in the material ofwaveguides 86. The left and right output couplers 88 may respectivelycouple the left and right images from the left and right waveguides 86towards the left and right eye boxes 60 for viewing by the user. Thisallows a user to view a computer-generated image (display image) such ascomputer-generated object 92 overlaid over real-world objects such asreal-world object 90.

By adjusting lenses and other optical components in components 100and/or 102, the distance from device 10 at which display image 92 is infocus for the user viewing from eye boxes 60 can be adjusted. Theseadjustments may be made without affecting the focus of real-worldobjects such as real-world object 90. In this way, real-world objectssuch as real-world object 90 may be observed by the user as if device 10were not present (e.g., without any intervening optical components)while computer-generated content such as virtual object 92 may be placedwithin the scene being viewed by the user at one or more desireddistances from the user.

Time-interleaving and polarization control techniques may be used inmerging real-world content and display content in the optical system fordevice 10.

Consider, as an example, the time-division multiplexing arrangement ofFIG. 3A. FIG. 3A is a diagram of an illustrative optical system (opticalsystem 122) that may be used for both the left-hand and right-handportion of device 10. As shown in FIG. 3A, system 122 includes outeroptical components such as optical components 100 and inner opticalcomponents such as optical components 102. Electrically adjustabledevices in components 100 and/or 102 are controlled by control circuitry12. Waveguide 86 and, in particular, the portion of waveguide 86 withoutput coupler 88, is interposed between components 100 and 102.Real-world image light 104 passes through system 122 and is viewable bya user's eye at eye box 60. Computer-generated image light (displaylight) 124 is guided to output coupler 88 through waveguide 86 to outputcoupler 88. Output coupler 88 couples light 124 out of waveguide 86, sothat light 124 passes through components 102 to eye box 60.

System 122 has bias lenses 106 and 120. The powers of bias lenses 106and 120 may be complementary. For example, bias lens 106 may have apositive lens power such as 1.5 diopter and bias lens 120 may have anegative lens power such as a −1.5 diopter. With this type ofarrangement, the positive power of lens 106 is cancelled by thecorresponding negative power of lens 120, so that the net effect is asif there were no lens present between the real-world objects and eye box60 (e.g., real-world image 104 experiences a zero lens power from lenses106 and 120 when traveling to eye box 60). At the same time, thenegative power of lens 120 is present in components 102.

Components 100 include electronic shutter 105. Electronic shutter 105,which may sometimes be referred to as an intensity switch orelectrically adjustable light modulator, may include linear polarizer108, polarization switch 110, and linear polarizer 112. Linear polarizer108 may have a pass axis aligned with the Y axis, so that light 104 islinearly polarized along the Y axis after passing through polarizer 108.Polarization switch 110, which may sometimes be referred to as anelectrically adjustable wave plate, electrically adjustable retarder, orelectrically adjustable polarization controller, may be formed from anelectrically adjustable optical component such as a twisted nematicliquid crystal layer (as an example). Alternating-current drive signalsmay be used to control the operation of polarization switch 110 to avoidundesirable charge accumulation effects that might otherwise arise fromusing a control signal of a fixed polarity.

In a first state (sometimes referred to as an OFF state, where a 0Vpeak-to-peak drive signal is applied), polarization switch 110 rotatesthe polarization of incoming linearly polarized light from polarizer 108by 90° so that light 104 is polarized along the X axis after exitingpolarization switch 110. Linear polarizer 112 has a pass axis alignedwith the X axis and therefore passes light 104 in the first state. In asecond state (sometimes referred to as an ON state, where a 20Vpeak-to-peak drive signal or other suitable drive signal is applied),polarization switch 110 does not rotate the polarization of incominglinearly polarized light. In this state, light 104 is blocked bypolarizer 112. As this demonstrates, the adjustability of polarizationswitch 110 allows polarizer 108, polarization switch 110, and polarizer112 to serve as an electrically adjustable shutter that can either blockor pass real-world light 104 to eye box 60.

Optical components 102 may include linear polarizer 114. Linearpolarizer 114 may have a pass axis aligned with the X axis and may serveto block light polarized along the Y axis as described in connectionwith polarizer 112. The inclusion of polarizer 112 may help reducedisplay light that leaks out of output coupler 88 in the +Z direction.If desired, polarizer 112 may be omitted. In configurations in whichpolarizer 112 is omitted, polarizer 114, polarizer 108, and polarizationswitch 110 form the electronic shutter.

Optical components 102 may include a tunable lens such as a liquidcrystal lens. The position of the lens center of the tunable lens and/orthe lens power of the lens may be adjusted by control circuitry 12. Forexample, the position of the lens center of the adjustable lens may becontrolled in real time based on information from a gaze tracking systemthat is monitoring the direction of gaze of the user (e.g., bymonitoring the user's eye in eye box 60). This allows the center of thelens to be aligned along the user's direction of gaze.

In the example of FIG. 3A, components 102 include liquid crystal lens118. Lens 118 is electrically adjustable. During operation, controlcircuitry 12 can adjust the power of liquid crystal lens 118 to placevirtual objects in desired focal planes. Liquid crystal lens 118 may, asan example, exhibit a positive lens power that is tunable between firstand second positive lens power values, may exhibit a negative lens powerthat is tunable between first and second negative lens power values, ormay have a lens power that is adjustable between a positive value (e.g.,+1 diopter) and a negative value (e.g., −1 diopter). The net power ofthe lens system between output coupler 88 and eye box 60 is given by thecombined lens powers of inner (rear) bias lens 120 and liquid crystallens 118. In some configurations, inclusion of a negative rear bias lensmay help provide a desired overall negative lens power to the user(e.g., a lens power ranging from −0.5 diopter which may be used to placevirtual objects at a focal plane distance of 2 m from eye box 60 to −2.5diopter which may be used to place virtual objects at a focal planedistance of 40 cm from eye box 60) while allowing lens 118 to exhibitboth positive and negative lens powers, thereby helping to avoid tuningchallenges that may sometimes be present when creating only negativeliquid crystal lens powers. Lens 118 may be configured to exhibit adesired lens power (e.g., 1 diopter and/or other suitable lens powers)for light polarized along the Y axis while exhibiting no lens power (0diopter) for light polarized along the X axis. Lens 118 may have anysuitable number of layers of liquid crystal cells (e.g., one layer, twoor more layers, three or more layers, etc.). If desired, amultiple-layer configuration may be used for lens 118 to allow lens 118to provide an electrically adjustable lens center position, to allow theoptical performance of lens 118 to be enhanced, etc.

Time division multiplexing may be used by optical system 122 to mergereal-world light 104 and display light 124 at eye box 60 for viewing bya user.

During first time periods, which may sometimes be referred to as “worldview off” periods, polarization switch 110 of intensity switch 105 isadjusted to block real-world light 104. Display light 124 from outputcoupler 88 is linearly polarized along the X axis by polarizer 114.Optical system 122 may have a polarization switch such as polarizationswitch 116. Polarization switch 116 may be turned OFF wheneverpolarization switch 110 is ON and intensity switch 105 is blockingreal-world image light 104. Because polarization switch 116 is OFF,polarization switch 116 rotates the polarization of display light 124 sothat display light 124 is aligned along the Y axis. Liquid crystal lens118 is adjusted by control circuitry 12 to produce a desired lens powerfor light polarized along the Y axis. Bias lens 120 provides additionaldesired lens power. Light 124 therefore reaches eye box 60 with adesired lens power interposed between optical coupler 88 and eye box 60.By adjusting this lens power (e.g., when control circuitry 12 adjustslens 118) while producing synchronized image frames with display 14while intensity switch 105 is opaque and blocking real-world light,virtual objects associated with respective frames of display image light124 may be placed in one or more desired focal planes.

During second time periods, which may sometimes be referred to as “worldview on” periods, polarization switch 110 of intensity switch 105 isadjusted to pass real-world light 104 while display 14 is optionallyturned off (and light 124 is not produced). Polarization switch 116 isplaced in a state that allows light to pass through lens 118. During thesecond time periods, the user views real-world objects through system122. Liquid crystal lens 118 is only sensitive to light polarized alongthe Y axis and is insensitive to light polarized along the X axis. Light104 is polarized along the X axis when passing through polarizer 114 andpolarization switch 116 may be turned ON, so light 104 maintains itspolarization state along the X-axis when passing through lens 118 and istherefore not affected by lens 118. The combined optical powers of frontbias lens 106 and rear bias lens 120 cancel (in this example), so thatno net lens power is present between eye box 60 and the real world(i.e., real-world light 104 reaches eye box 60 unaffected by opticalsystem 122). As shown in FIG. 3B, while real-world light 104 passesthrough system 122 in the second time periods (world view on periods),display 14 can be either turned on or off. Display 14 may, for example,be turned on or off depending on the depth of the virtual content to bedisplayed for the user in eye box 60. For example, display 14 can beturned on if the virtual content that is to be placed at the focal planecorresponding to lens 118 has zero optical power.

During operation, control circuitry 12 may operate the polarizationswitches and other adjustable components of device 10 synchronization(e.g. alternating between world view on and world view off periods). Therelative duty cycle between the world view on and off states may be 50%(50% on and 50% off) or may have any other suitable value (e.g., 60%-70%on, less than 80% on, more than 30% on, etc.). The world view may alsobe on with 100% duty cycle when there is no need to adjust the depth ofthe virtual content. In other words, in the configuration of FIGS. 3Aand 3B (and, if desired, other configurations such as the configurationsof FIGS. 6 and 9 ), the depth feature can be disabled when it is desiredto increase the brightness of the world view.

If desired, other polarization-dependent lenses may be used for lens118. For example, a geometrical phase lens or a fixed birefringent lensmay be used in place of tunable lens 118. A fixed polarization-dependentlens provides two different lens power choices to system 122, dependingon the polarization state of the light passing through the lens. Eyetracking and lens center adjustments are not be used in thisconfiguration, because the lens center positions of the fixed lens arefixed.

If desired, optical system 122 may use pairs of complementarygeometrical phase lenses. Geometrical phase lenses may be implementedusing liquid crystal lens structures configured to exhibit a positivelens power for one circular polarization such as right-hand circularpolarization (RCP) and a negative lens power for an opposite circularpolarization such as left-hand circular polarization (LCP). Because bothpositive and negative lens powers are exhibited when presented withunpolarized light (containing equal portions of RCP and LCP light),polarization control is used to avoid undesired double images.

FIG. 4A shows an optical system based on geometrical phase lenses GPL1and GPL2. Front and rear bias lenses 106 and 120 are omitted from FIG.4A and the subsequent FIGS. to avoid over-complicating the drawings.

As shown in FIG. 4A optical system 122 may have quarter wave plate QWP2between geometrical phase lens GPL2 and polarization switch P2 (anelectrically adjustable polarization rotator). Linear polarizer LPOL isinterposed between waveguide 86 (output coupler 88) and polarizationswitch P1. Quarter wave plate QWP1 is located between polarizationswitch P1 and lens GPL1. In this configuration, polarization switches P1and P3 operate as polarization rotators for linear polarization. Inother configurations, quarter wave plate QWP2 and polarization switch P2can be combined to form a different type of polarization switch whichworks for circular polarization. Similarly, quarter wave plate QWP1 andpolarization switch P1 can be combined to form a polarization switchthat works for circularly polarized light. In other words, quarter waveplates QWP1 and QWP2 can be omitted.

When polarization switch P2 and polarization switch P1 are OFF, RCPreal-world light 104 is converted to LCP light by lens GPL2. Quarterwave plate QWP2 converts this LCP light to light that is linearlypolarized along the Y axis. Polarization switch P2 is off and thereforerotates this light so that it is polarized along the X axis. Linearpolarizer linear POL blocks this light. In this way, RCP real-worldlight is prevented from reaching the user.

When polarization switch P2 and polarization switch P1 are OFF, LCPreal-world light 104 is converted to RCP light by lens GPL2, whichexhibits a negative lens power. This light is converted to linearlypolarized light that is polarized along the X axis by quarter wave plateQWP2. Polarization switch P2 is OFF and therefore rotates thepolarization of this light so that it is linearly polarized along the Yaxis. After passing through waveguide 86 (output coupler 88) and linearpolarizer LPOL, this light reaches polarization switch P1. Polarizationswitch P1 is OFF and therefore rotates the polarization of light 104 sothat the light exiting polarization switch P1 is polarized along the Xaxis. Quarter wave plate QWP1 converts this linearly polarized light toRCP light. As the RCP light passes through lens GPI, lens GPI exhibits apositive lens power equal and opposite to that of lens GPL2, soreal-world light 104 is not affected by any lens power (e.g., the lenspower of lenses GPL2 and GPL1 when combined is 0 diopter, so thatreal-world light 104 can be viewed by the user as if system 122 were notpresent).

When polarization switches P and P2 are ON, LCP light 104 is blocked.RCP light passes through lens GPL2, which exhibits a positive lenspower. Polarization switches P1 and P2 are ON and therefore do notchange the polarization state of the light passing through them. Afterpassing through the components between lens GPL2 and GPL1, light 104becomes left-hand circularly polarized. As shown in FIG. 4A, when LCPlight 104 reaches lens GPL1, lens GPL1 exhibits a negative lens powerequal and opposite to the positive lens power of lens GPL2. As whenpolarization switches P1 and P2 were both OFF, real-world light 104 isunaffected by the presence of lenses GPL1 and GPL2 when polarizationswitches P1 and P2 are ON, because the lens powers of lenses GPL1 andGPL2 cancel each other.

Display light 124, in contrast, is affected by the switching ofpolarization switches P1 and P2. When these switches are OFF, light 124is RCP at the input to lens GPL1, which exhibits a positive lens power.When polarization switches P1 and P2 are ON, however, light 124 is LCPat the input to lens GPL1, so that lens GPL1 exhibits a negative lenspower.

During operation, the states of polarization switches P1 and P2 areadjusted in tandem (e.g., by alternating between ON and OFF insynchronization with each other with a desired duty cycle). Real-worldlight 104 is unaffected by the changes in state of polarization switchesP1 and P2, which allows the user to view the real world through system122 as if system 122 were not present. Display light 124 experiencesalternating lens powers due to its changing polarization state. Whenconverted to RCP light, lens GPL1 applies a positive lens power todisplay light 124, whereas when converted to LCP light, lens GPL1applies a negative lens power to display light 124. The system of FIG.4A therefore allows virtual objects (e.g., display image frames fromdisplay 14 that are appropriately synchronized with the switching ofpolarization switches 1 and 2) to be placed at two different focalplanes. Systems such as the system of FIG. 4A and the other systemsdescribed herein may, if desired, use component stacking arrangements toimplement additional levels of focal plane positioning. The arrangementswhere optical system 122 exhibits first and second states with first andsecond respective focal plane positions for virtual objects in displayimages are described herein as an example.

In the illustrative configuration of FIG. 4B, intensity switch 105 has apair of linear polarizers LPOL and a switchable polarizer P2. A linearpolarizer LPOL, polarization switch P1 (for selecting a desired lenspower for geometric phase lens GPL), and quarter wave plate QWP arelocated between geometric phase lens GPL and waveguide 86. With thisconfiguration, the power of the bias lens can be configured such thatdisplay 14 may exhibit slight negative power when the positive power ofgeometric phase lens GPL is selected. In an example, geometric phaselens GPL has a power of +/−1D, negative bias lens 120 has a power of−1.5D, and positive bias lens 106 has a power of 0.5D. When the positivepower is selected for geometric phase lens GPL, display power will be+1D−1.5D=−0.5D and the power applied to real-world light (sometimesreferred to as world power) will be +0.5D+1D−1.5D=0D. When negativepower is selected, display power will be −1D−1.5D=−2.5D.

In the illustrative configuration of FIG. 5A, optical system 122 hasonly a single polarization switch (switch P) and the location of linearpolarizer LPOL has been changed so that linear polarizer LPOL is betweenpolarization switch P and eye box 60. There is also only a singlequarter wave plate (QWP) in system 122 of FIG. 5A. As shown in FIG. 5A,real-world light 104 passes through system 122 with 0 lens powerregardless of whether polarization switch P is ON or OFF, because thepositive power of geometrical phase lens GPL2 cancels the equal andopposite negative lens power of geometrical phase lens GPL1, whereasdisplay light 124 passes through a negative lens (GPL1) whenpolarization switch P is OFF and passes through a positive lens (GPL1)when the polarization switch is ON.

In the illustrative configuration of FIG. 5B, a clean-up polarizationswitch formed from linear polarizer LPOL′, polarization switch P′, andquarter wave plate QWP′ has been added to optical system 122 of FIG. 5Ato help block undesired ghost image light and thereby improve thecontrast ratio between the main image and ghost image. Polarizationswitch P′ may be operated in synchronization with polarization switch P.

In the illustrative configuration of FIG. 6 , system 122 has apolarization switches PSA and PSB. Switch PSA may be used with a pair oflinear polarizers LPOLX and LPOLY to implement an intensity switch.Switch PSB may be configured so that when switch PSB is ON, RCP lightpasses through switch PSB unaffected and may be configured so that whenswitch PSB is OFF, incoming RCP light is converted to LCP light.

During operation in “world view on” mode, display 14 is turned off andswitch PSA (and the electronic switch formed from polarizers LPOLX andLPOLY and polarization switch PSA) may be adjusted to pass light 104through waveguide 86 and coupler 88. LCP light is presented to lensGPL1, which exhibits a negative lens power and RCP light is presented tolens GPL2, which exhibits a cancelling positive lens power.

During operation in “world view off” mode, display 14 is turned on andswitch PSA is adjusted to block real-world light 104. Frames of imagelight (e.g., alternating first and second frames corresponding toalternating first and second virtual objects) are synchronized with thestate of polarization switch PSB. When the first frames are presented,switch PSB is turned ON and light 124 experiences a negative lens powerwhen passing through lens GPL1 and a cancelling positive lens power whenpassing through lens GPL2. When the second frames are presented, switchPSB is turned OFF and light 124 experiences a negative lens power whenpassing through lens GPL1 and another negative lens power when passingthrough lens GPL2. In this way, the first frames of display lightexperience 0 lens power and the second frames of display lightexperience a negative lens power (equal to the summation of the negativelens powers of lenses GPL1 and GPL2). As with the other systems shown inthe FIGS., bias lenses such as a front positive fixed bias lens and acomplementary rear negative fixed bias lens may be included in system122.

Another illustrative arrangement for optical system 122 is shown in FIG.7 . When polarization switch P is ON, real-world light 104 experiences 0lens power and RCP display light 124 experiences 0 lens power. LCPdisplay light is blocked. When polarization switch P is OFF, real-worldlight is blocked by linear polarizer LPOL and display light 124experiences a negative lens power. Display 14 can be turned on and offdepending on desired content depth.

In the illustrative configuration of FIG. 8 , display 14 supplieswaveguide 86 alternately with RCP display light 124 or LCP display light124 and waveguide 86 preserves the polarization state of the light fromdisplay 14. Components such as the quarter wave plate QWP, linearpolarizer LPOL, and polarization switch P of FIG. 7 may be omitted.Different virtual object locations are achieved by displaying contentfor different depths with different polarizations (RCP for one depth andLCP for another depth).

FIG. 9 shows another illustrative configuration for optical system 122.In the example of FIG. 9 , geometrical phase lens (GPL′) has beenconfigured to pass 0^(th) order light (light striking lens GPL′ at anangle parallel to the surface normal of lens GPL′) with an intensitycomparable to that of the RCP and LCP output light from lens GPL′.Accordingly, when lens GPL′ of FIG. 9 is presented with unpolarizedlight (e.g., light having equal parts of LCP and RCP light), there willbe three outputs: 1) LCP light (experiencing a negative lens power), 2)RCP light (experiencing a positive lens power), and 3) 0^(th) orderlight (experiencing no lens power). Optical system 122 of FIG. 9 isconfigured to block LCP light exiting lens GPL′. When display 14 is off,polarization switch P2 (which forms an electronic shutter with linearpolarizers LPOL-1 and LPOL-2) is configured to allow real-world light topass to polarization switch P1. Polarization switch P1 is OFF, whichallows the real-world light to be passed to lens GPL′ as RCP light. Asreal-world light 104 passes through lens GPL′, lens GPL′ exhibits a 0lens power.

When display 14 is on, polarization switch P2 is adjusted so thatreal-world light 104 is blocked. The state of polarization switch P1 isalternated in synchronization with the image frames produced by display14, so that virtual objects can be presented in different focal planes.When switch P1 is ON, light 124 experiences a negative lens power whenpassing through lens GPL′ and when switch P is OFF, light 124experiences a 0 lens power when passing through lens GPL′.

System 8 may gather and use personally identifiable information. It iswell understood that the use of personally identifiable informationshould follow privacy policies and practices that are generallyrecognized as meeting or exceeding industry or governmental requirementsfor maintaining the privacy of users. In particular, personallyidentifiable information data should be managed and handled so as tominimize risks of unintentional or unauthorized access or use, and thenature of authorized use should be clearly indicated to users.

Physical Environment

A physical environment refers to a physical world that people can senseand/or interact with without aid of electronic systems. Physicalenvironments, such as a physical park, include physical articles, suchas physical trees, physical buildings, and physical people. People candirectly sense and/or interact with the physical environment, such asthrough sight, touch, hearing, taste, and smell.

Computer-Generated Reality

In contrast, a computer-generated reality (CGR) environment refers to awholly or partially simulated environment that people sense and/orinteract with via an electronic system. In CGR, a subset of a person'sphysical motions, or representations thereof, are tracked, and, inresponse, one or more characteristics of one or more virtual objectssimulated in the CGR environment are adjusted in a manner that comportswith at least one law of physics. For example, a CGR system may detect aperson's head turning and, in response, adjust graphical content and anacoustic field presented to the person in a manner similar to how suchviews and sounds would change in a physical environment. In somesituations (e.g., for accessibility reasons), adjustments tocharacteristic(s) of virtual object(s) in a CGR environment may be madein response to representations of physical motions (e.g., vocalcommands).

A person may sense and/or interact with a CGR object using any one oftheir senses, including sight, sound, touch, taste, and smell. Forexample, a person may sense and/or interact with audio objects thatcreate 3D or spatial audio environment that provides the perception ofpoint audio sources in 3D space. In another example, audio objects mayenable audio transparency, which selectively incorporates ambient soundsfrom the physical environment with or without computer-generated audio.In some CGR environments, a person may sense and/or interact only withaudio objects.

Examples of CGR include virtual reality and mixed reality.

Virtual Reality

A virtual reality (VR) environment refers to a simulated environmentthat is designed to be based entirely on computer-generated sensoryinputs for one or more senses. A VR environment comprises a plurality ofvirtual objects with which a person may sense and/or interact. Forexample, computer-generated imagery of trees, buildings, and avatarsrepresenting people are examples of virtual objects. A person may senseand/or interact with virtual objects in the VR environment through asimulation of the person's presence within the computer-generatedenvironment, and/or through a simulation of a subset of the person'sphysical movements within the computer-generated environment.

Mixed Reality

In contrast to a VR environment, which is designed to be based entirelyon computer-generated sensory inputs, a mixed reality (MR) environmentrefers to a simulated environment that is designed to incorporatesensory inputs from the physical environment, or a representationthereof, in addition to including computer-generated sensory inputs(e.g., virtual objects). On a virtuality continuum, a mixed realityenvironment is anywhere between, but not including, a wholly physicalenvironment at one end and virtual reality environment at the other end.

In some MR environments, computer-generated sensory inputs may respondto changes in sensory inputs from the physical environment. Also, someelectronic systems for presenting an MR environment may track locationand/or orientation with respect to the physical environment to enablevirtual objects to interact with real objects (that is, physicalarticles from the physical environment or representations thereof). Forexample, a system may account for movements so that a virtual treeappears stationery with respect to the physical ground.

Examples of mixed realities include augmented reality and augmentedvirtuality.

Augmented Reality

An augmented reality (AR) environment refers to a simulated environmentin which one or more virtual objects are superimposed over a physicalenvironment, or a representation thereof. For example, an electronicsystem for presenting an AR environment may have a transparent ortranslucent display through which a person may directly view thephysical environment. The system may be configured to present virtualobjects on the transparent or translucent display, so that a person,using the system, perceives the virtual objects superimposed over thephysical environment. Alternatively, a system may have an opaque displayand one or more imaging sensors that capture images or video of thephysical environment, which are representations of the physicalenvironment. The system composites the images or video with virtualobjects, and presents the composition on the opaque display. A person,using the system, indirectly views the physical environment by way ofthe images or video of the physical environment, and perceives thevirtual objects superimposed over the physical environment. As usedherein, a video of the physical environment shown on an opaque displayis called “pass-through video,” meaning a system uses one or more imagesensor(s) to capture images of the physical environment, and uses thoseimages in presenting the AR environment on the opaque display. Furtheralternatively, a system may have a projection system that projectsvirtual objects into the physical environment, for example, as ahologram or on a physical surface, so that a person, using the system,perceives the virtual objects superimposed over the physicalenvironment.

An augmented reality environment also refers to a simulated environmentin which a representation of a physical environment is transformed bycomputer-generated sensory information. For example, in providingpass-through video, a system may transform one or more sensor images toimpose a select perspective (e.g., viewpoint) different than theperspective captured by the imaging sensors. As another example, arepresentation of a physical environment may be transformed bygraphically modifying (e.g., enlarging) portions thereof, such that themodified portion may be representative but not photorealistic versionsof the originally captured images. As a further example, arepresentation of a physical environment may be transformed bygraphically eliminating or obfuscating portions thereof.

Augmented Virtuality

An augmented virtuality (AV) environment refers to a simulatedenvironment in which a virtual or computer generated environmentincorporates one or more sensory inputs from the physical environment.The sensory inputs may be representations of one or more characteristicsof the physical environment. For example, an AV park may have virtualtrees and virtual buildings, but people with faces photorealisticallyreproduced from images taken of physical people. As another example, avirtual object may adopt a shape or color of a physical article imagedby one or more imaging sensors. As a further example, a virtual objectmay adopt shadows consistent with the position of the sun in thephysical environment.

Hardware

There are many different types of electronic systems that enable aperson to sense and/or interact with various CGR environments. Examplesinclude head mounted systems, projection-based systems, heads-updisplays (HUDs), vehicle windshields having integrated displaycapability, windows having integrated display capability, displaysformed as lenses designed to be placed on a person's eyes (e.g., similarto contact lenses), headphones/earphones, speaker arrays, input systems(e.g., wearable or handheld controllers with or without hapticfeedback), smartphones, tablets, and desktop/laptop computers. A headmounted system may have one or more speaker(s) and an integrated opaquedisplay. Alternatively, a head mounted system may be configured toaccept an external opaque display (e.g., a smartphone). The head mountedsystem may incorporate one or more imaging sensors to capture images orvideo of the physical environment, and/or one or more microphones tocapture audio of the physical environment. Rather than an opaquedisplay, a head mounted system may have a transparent or translucentdisplay. The transparent or translucent display may have a mediumthrough which light representative of images is directed to a person'seyes. The display may utilize digital light projection, OLEDs, LEDs, pLEDs, liquid crystal on silicon, laser scanning light source, or anycombination of these technologies. The medium may be an opticalwaveguide, a hologram medium, an optical combiner, an optical reflector,or any combination thereof. In one embodiment, the transparent ortranslucent display may be configured to become opaque selectively.Projection-based systems may employ retinal projection technology thatprojects graphical images onto a person's retina. Projection systemsalso may be configured to project virtual objects into the physicalenvironment, for example, as a hologram or on a physical surface.

The foregoing is merely illustrative and various modifications can bemade to the described embodiments. The foregoing embodiments may beimplemented individually or in any combination.

What is claimed is:
 1. An electronic device, comprising: a waveguidehaving an output coupler; a display configured to provide a displayimage to the waveguide that is coupled out of the waveguide towards aneye box by the output coupler; a polarization-dependent lens throughwhich a real-world image for a real-world object is viewable; anelectronic shutter, wherein the electronic shutter is configured toswitch between a clear state in which the real-world image passesthrough the electronic shutter and an opaque state in which thereal-world image is blocked by the electronic shutter; and apolarization switch between the output coupler and thepolarization-dependent lens, wherein the polarization switch isconfigured to alternately switch between: a first state when theelectronic shutter is in the clear state, wherein in the first state thepolarization switch is configured to allow the real-world image to passthrough the polarization-dependent lens to the eye box without beingfocused by the polarization-dependent lens; and a second state when theelectronic shutter is in the opaque state, wherein in the second statethe polarization switch is configured to allow the display image to befocused by the polarization-dependent lens while passing to the eye box.2. The electronic device defined in claim 1 wherein the real-world imagehas a first linear polarization upon reaching the polarization switchand wherein the display image has the first linear polarization uponreaching the polarization switch, and wherein the polarization switch isconfigured to: pass the real-world image through the polarization switchwith the first linear polarization when the polarization switch is inthe first state; and pass the display image through the polarizationswitch while rotating the polarization of the display image from thefirst linear polarization to a second linear polarization that isorthogonal to the first linear polarization when the polarization switchis in the second state.
 3. The electronic device defined in claim 2wherein the polarization-dependent lens comprises a liquid crystal lens.4. The electronic device defined in claim 2 wherein thepolarization-dependent lens comprises a liquid crystal lens that isconfigured to focus light with the second linear polarization and isconfigured to not focus light with the first linear polarization.
 5. Theelectronic device defined in claim 4 wherein the electronic shuttercomprises an electrically adjustable liquid crystal layer.
 6. Theelectronic device defined in claim 5 wherein the electronic shuttercomprises first and second linear polarizers.
 7. The electronic devicedefined in claim 6 wherein the electrically adjustable liquid crystallayer is between the first and second linear polarizers.
 8. Theelectronic device defined in claim 7 wherein the output coupler isbetween the electrically adjustable liquid crystal layer and the secondlinear polarizer.
 9. The electronic device defined in claim 7 whereinthe second linear polarizer is between the electrically adjustableliquid crystal layer and the output coupler.
 10. The electronic devicedefined in claim 1 wherein the polarization-dependent lens comprises afixed birefringent lens that is not electrically adjustable.
 11. Theelectronic device defined in claim 1 wherein the polarization-dependentlens comprises an electrically adjustable lens.
 12. The electronicdevice defined in claim 1 further comprising a positive bias lens and anegative bias lens, wherein the electronic shutter, the output coupler,the polarization switch, and the polarization-dependent lens are betweenthe positive bias lens and the negative bias lens.
 13. A head-mounteddevice, comprising: a display configured to output a display image; awaveguide with an output coupler configured to pass the display imagefrom the display to an eye box; a polarization-dependent lens interposedbetween the output coupler and the eye box; a first polarization switchbetween the polarization-dependent lens and the output coupler; and anelectrically adjustable shutter having first and second linearpolarizers and a second polarization switch between the first and secondlinear polarizers, wherein the second polarization switch is switchablebetween a first state in which the electrically adjustable shutterblocks a real-world image from a real-world object from reaching thepolarization-dependent lens and a second state in which the electricallyadjustable shutter passes the real-world image to thepolarization-dependent lens.
 14. The head-mounted device defined inclaim 13 wherein the first polarization switch is configured to switchbetween first and second states in corresponding synchronization withthe first and second states of the second polarization switch.
 15. Thehead-mounted device defined in claim 14 wherein the display isconfigured to turn off the display image when the second polarizationswitch is in the second state and is configured to turn on the displayimage when the second polarization switch is in the first state.
 16. Thehead-mounted device defined in claim 15 wherein the display imagereaches the first polarization switch with a first linear polarizationwhen the second polarization switch is in the first state and whereinthe first polarization switch is configured to rotate the first linearpolarization of the display image to a second linear polarization thatis orthogonal to the first linear polarization when the firstpolarization switch is in the first state.
 17. The head-mounted devicedefined in claim 16 wherein the real-world image reaches the firstpolarization switch with the first linear polarization when the secondpolarization switch is in the second state and wherein the firstpolarization switch is configured to pass the real-world image of thefirst linear polarization without rotating the first linear polarizationof the real-world image.
 18. A head-mounted device, comprising: adisplay; switchable optical components configured to switchsynchronously between a first state and a second state; and a tunableliquid crystal lens through which a display image from the display isprovided to an eye box during the first state and not the second stateand through which a real-world image from a real-world object isprovided to the eye box during the second state and not the first state,wherein the switchable optical components comprise a polarization switchthrough which the display image is provided to the eye box during thefirst state.
 19. The head-mounted device defined in claim 18 wherein theswitchable optical components comprise: first and second linearpolarizers; and an additional polarization switch between the first andsecond linear polarizers, wherein the polarization switch is between thesecond linear polarizer and the tunable liquid crystal lens.
 20. Thehead-mounted device defined in claim 19 further comprising: a waveguidewith an output coupler that is configured to pass the display image fromthe display to the polarization switch.
 21. The head-mounted devicedefined in claim 18 wherein the tunable liquid crystal lens has anelectrically adjustable lens center position.
 22. An electronic device,comprising: a display configured to provide a display image toward aneye box; a polarization switch through which the display image isviewable from the eye box; a polarization-dependent lens through which areal-world image is viewable from the eye box; and an electronicshutter, wherein the electronic shutter is configured to switch betweena first state in which the real-world image passes through theelectronic shutter and a second state in which the real-world image isblocked by the electronic shutter.
 23. The electronic device defined inclaim 22, wherein the display is configured to provide the display imagetoward the eye box via an output coupler and wherein the polarizationswitch is between the output coupler and the polarization-dependentlens.
 24. An electronic device, comprising: a display configured toprovide a display image; a polarization switch through which the displayimage is viewable; an electronic shutter that overlaps the polarizationswitch, wherein the electronic shutter comprises an additionalpolarization switch and wherein the polarization switch is configured toswitch between first and second states in synchronization with first andsecond states of the additional polarization switch of the electronicshutter; and a polarization-dependent lens that overlaps thepolarization switch, wherein a real-world image is viewable through theelectronic shutter and the polarization-dependent lens when thepolarization switch is in the second state and the additionalpolarization switch is in the second state.